Digital cameras have an inherent limit to their spatial resolution that is governed by the optical lens and CCD array. To improve the image quality, super-resolution can be utilized by fusing multiple low-resolution images of the same scene to produce a relatively high-resolution image.
In recent years, various super-resolution techniques have been developed for estimating a high-resolution image from a set of low-resolution images. It was demonstrated early on that the aliasing effects in the low-resolution images can be removed and the high-resolution fused image is recovered, as long as there existed a sub-pixel motion in the low-resolution input images. Even though the relatively clean frequency domain description of super-resolution provided near desired results for very simple imaging scenarios, it was evident that super-resolution in general is computationally complex and numerically ill-behaved, necessitating more sophisticated super-resolution methods be developed.
It is important to note that almost all super-resolution methods been designed to increase the resolution of a single channel (monochromatic) image, and to date there is very little work addressing the problem of color super-resolution. In addressing color super-resolution, one method uses a set of previously demosaiced color low-resolution frames and fuses them together to enhance their spatial resolution. The typical solution involves applying monochromatic super-resolution algorithms to each of the color channels independently, while using the color information to improve the accuracy of motion estimation. Another approach is transforming the problem to a different color space, where chrominance layers are separated from luminance, and super-resolution is applied only to the luminance channel. Both of these methods are suboptimal as they do not fully exploit the correlation across color bands, where ignoring the relation between different color channels will result in color artifacts in the super-resolved images. Moreover, even proper treatment of the relation between the color layers is not sufficient for removing color artifacts if the measured images are mosaiced.
For demosaicing, a color image is typically represented by combining three separate monochrome images. Ideally, each pixel reflects three data measurements; one for each of the color bands. In practice, to reduce production costs, many digital cameras have only one color measurement (red, green, or blue) per pixel. The detector array is a grid of CCD's, each made sensitive to one color by placing a color-filter array (CFA) in front of the CCD. The Bayer pattern is a very common example of such color filter. The values of the missing color bands at every pixel are often synthesized using some form of interpolation from neighboring pixel values, to estimate the underdetermined color values. This process is know as demosaicing.
Numerous demosaicing methods have been proposed through the years to solve the under-determination problem. Linear interpolation of known pixel values applied to each color band independently is one method to estimate the unknown pixel values. This approach does not consider some important information about the correlation between the color bands and results in substantial color artifacts. Because the Bayer pattern has two times the number of green pixels than the red or blue pixels, the red and blue channels are down-sampled two-times more than the green channel. Therefore, the independent interpolation of the green band will result in a more reliable reconstruction than the red or blue bands. From this, with the assumption that the red/green and blue/green ratios are similar for the neighboring pixels, the basics of the smooth hue transition method evolved.
There is a negligible correlation between the values of neighboring pixels located on the different sides of an edge in an image. Although the smooth hue transition method is logical for smooth regions of the reconstructed image, it is not useful for the high-frequency (edge) areas. Consequently, gradient-based methods were applied but did not perform interpolation across the edges of an image, where this non-iterative method uses the second derivative of the red an blue channel to estimate the edge direction in the green channel, and the green channel is then used to compute the missing values in the red and blue channels.
A modified gradient-based method was subsequently developed, where the second derivative of the green channel and the first derivative of the red (or blue) channels are used to estimate the edge direction in the green channel. The smooth hue method was later combined to provide an iterative method where the smooth hue interpolation is done with respect to the local gradient computed in eight directions about the pixel of interest. A second stage using anisotropic inverse diffusion further enhanced the quality of the reconstructed image. This two-step approach of interpolation followed by an enhancement step has been widely adopted, where spatial and spectral correlations among neighboring pixels are exploited to define the interpolation step, while adaptive median filtering is used as the enhancement step. Other iterative implementation methods of the median filters have been used as the enhancement step that take advantage of the homogeneity assumption in the neighboring pixels.
Iterative maximum a posteriori (MAP) methods are another important category of demosaicing methods. A MAP algorithm with a smooth chrominance prior has been developed, where the original image is transformed to the YIQ representation. The chrominance interpolation is preformed using isotropic smoothing. The luminance interpolation is done using edge directions computed in steerable wavelet pyramidal structure.
Almost all of the demosaicing methods are based on one or more of the following assumptions.                1) In the constructed image with the mosaicing pattern, there are more green sensors with regular pattern of distribution that blue or red ones (in the case of Bayer CFA, there are twice as many greens than red of blue pixels and each is surrounded by four green bands.        2) Most algorithms assume a Bayer CFA pattern, for which each red, green, and blue pixel is a neighbor to pixels of different color bands.        3) For each pixel, one and only one, color band value is available.        4) The pattern of pixels does not change through the image.        5) The human eye is more sensitive to the details in the luminance component of the image than the details in the chrominance component.        6) The human eye is more sensitive to chrominance changes in the low spatial frequency region than the luminance change.        7) Different color bands are correlated with each other.        8) Edges should align between color bands.        
To date, the most sophisticated demosaicing methods have failed to produce satisfactory results when severe aliasing is present in the color-filtered image. Such severe aliasing occurs with inexpensive commercial still or video digital cameras having a small number of CCD pixels, where the color artifacts worsens as the number of CCD pixels decreases.
The poor quality of single-frame demosaiced images necessitates the need for improved multi-frame methods, where the information of several low-quality images are fused together to produce high-quality demosaiced images.
Accordingly, there is a need to develop more effective and efficient methods of image reconstruction to overcome the current shortcomings in the art.